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Abstract 


The validity of peer ratings as a supplemental measure 
to the Marine Corps' performance evaluation system has been 
examined and subjected to analysis by the multitrait- 
multirater method. In a Marine Corps rifle company, two 
Supervisory levels rated subordinates, and the subordinates 
Mimeuinetated one another. Results revealed neither con- 
vergent validity (agreement among the raters) nor discriminant 
validity (independence of the rating scales). Results did 
provide evidence of disagreement among the raters on the 
performance of those being rated. Thus, official use of 
peer ratings in a performance evaluation system is question- 
able; however, an unofficial use to gain viewpoints of 


different levels of an organization may have merit. 
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Introduction 

Procedures for the performance evaluation of all Marine 
officers and non-commissioned officers (NCO's) in the grade 
of Sergeant and above are outlined in Marine Corps Order 
1610.7A. The order states "the fitness report is an evalua- 
tion of duties performed, represents a comprehensive 
portrayal of the professional qualifications, personal traits, 
and potential of the individual, and is vital in determining 
duty assignments and selection for promotion ee Ue: 

One problem persisting in evaluating officer and NCO 
performance is the status of those doing the evaluation 
relative to those being evaluated. In 1947, Williams and 
Leavitt conducted a study using as subjects officer candidates 
at the Marine Corps! Officer Candidate School. The results 
of their study indicated that peer evaluations provided more 
valid measures of those being evaluated than did superior 
evaluations, because the peers had more time to observe one 
MOoClcteaitimuneciiesiperror officers had, the peers knew one 
another in their real environment, and the peers reacted 
more realistically to one another's behavior (Williams and 
Leavitt, 1947). What these results mean is that peers ob- 
serve one another behave in a working environment when 
SUpertoOroea@on t. the validity of peer evaluations as a 
predictor of the peers eventual success has been investigated 
Poeiomemninbitary settings, too, (Hollander, 1954 & 1965; 
Pomtibanelooo; Reynolds, 1966; Wherry and Fryer, 1949; Wollack 


ailment tman, 1961). 





The military is not the only setting in which peer 
nomination studies have been conducted. Industry has examined 
peer nominations and their usefulness at managerial levels 
(Dent and Mann, 1954; Roadman, 1964; Weitz, 1958). Findings 
from industry showed that peers can identify certain charac- 
teristics among their fellow peer supervisors which differ- 
entiate those considered promotable by higher level management 
from those considered not promotable (Weitz, 1958). The 
ecucaeional field, too, has investigated peer nominations 
(Cassel and Martin, 1964; Smith, 1967). In all these studies, 
the period which elapsed from the time of the initial rating 
until the final performance level (criterion) was measured, 
varied from several months (Wherry and Fryer, 1949) up to 
7 consmisoleander, NI65)pandmthe validity coeffaements re- 
ported typically ranged around .40, though some higher 
coefficients have also been found (Reynolds, 1966). 

But the real value of peer evaluations may go beyond 
these findings in military, industrial, and educational 
settings. The Marine Corps is built on teamwork. To the 
extent that the output of the Marine Corps' teamwork is 
efficient, effective, and individually rewarding, the Marine 
Corps will accomplish its missions in such a way as to maximize 
JObmPciaOrmanee and job satisfaction. The fitness report, 
indeed, is a measure of the individual's performance in the 
Manane (orps, and it appears that the findings of military, 
imcaucerial wand educational studies support the use of peer 
ratings in a performance evaluation system. The purpose of 


this study is to examine the use of peer evaluations as a 





means of supplementing the current Marine Corps performance 
evaluation system. 

Method 
BHD )}EeECcts 

The study was conducted in the lst Marine Division, 

Sanp Pendleten, California, during the summer of 1974. The 
population consisted of Marines in a randomly selected rifle 
company. Two supervisory levels of Reporting Seniors (RS's) 
each evaluated Squad Leaders under their command, and the 
Squad Leaders (P's) in turn provided peer evaluations of one 
aaomener. A Peer Evaluation Form (PEF) developed by the in- 
wesereator was used to conduct the evaluation. This form is 
shown in Appendix A. 

The two supervisory levels consisted of one Company 
Commander (RS 2), a Captain in rank with seven years of 
Service, including a recent combat tour, and four Platoon 
Commanders (RS 1's), who were Lieutenants with an average 
Gil. 5 years Of service. 

The peers consisted of twelve Squad Leaders (P's), all 
of whom were Corporals with an average time in service of 
2.8 years, an average time in grade of 4.2 months, and an 
dvemapeeace of 22 years. 

Materials 

The RS's and the P's used an evaluation form in conduct- 
ing their performance evaluations. Three rating scales in 
the present Marine Corps Fitness Report ("Regular Duties" 
foecme wee’ General Value to the Service" (Scale 2), and 


"Requirements in War" (Scale 3)) were selected as the evaluation 





scales. The rating levels of each of these scales was used 

aS in the present Marine Corps Fitness Report (Outstanding, 
Excellent, Above Average, Average, Below Average, Unsatisfactory, 
and Not Observed). In order to standardize the scales and 

the rating levels among the RS's and P's, instructions de- 

fining the scales and the rating levels were provided on the 
evaluation form. Appendix A is a sample of the form. 

PGocealye 

Before gathering both the RS's and the P's evaluations, 
Mitmilvesticator met with all subjects to review the instruc- 
tions contained on the evaluation form. The subjects then 
completed their evaluations. Each subject, upon completion 
of his evaluations, independently placed them in a collection 
box provided. After the last subject had completed his 
evaluations, the investigator gathered all forms from the 
ea lleceron box. 

To derive total RS and P scores on the evaluation form, 
numbers were assigned to the rating scale levels. The 
highest rating, Outstanding (0S), was awarded a plus 5, the 
next highest, Excellent (EX), was awarded a plus 4, and so 
on through the rating level Unsatisfactory (UN), which was 
awarded a 0. The rating Not Observed was recorded as "NO." 
A sum for each P was then obtained by adding the numerical 
ratings given him by all other P's, yielding a positive one 
imewomwadieit score that reflected the total score of the P 
being evaluated. Figure I is an example showing how such a 


Seore was calculated. 
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Figure I 


Peonple of alculationsof the Total Score 
LOn ead Heer 


Rating Levels 
(points awarded in parentheses) 


NO UN BA AV AA EX OS 
No!) (0) Se) 2) GB) ee 5) 


Scales 





Heotal Score tor the Peer = 3 + 2 + "NO" = 5. 

Figure I shows how the ratings were summarized when one 
peer had been rated by one rater (a peer, a RS 1 or RS 2). 
As a next step, all of the ratings of a peer by a level of 
Porers (peemomro 1, or RS 2) were Summarized. Figure I1 


displays how this summarization was conducted. 





Figure II 


Example of Rating Summarization Procedure 
for the Peers and Computed Average Ratings 
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foeolieacrlal data collection, 
Mer therefore, 
eBEeR ne CIE Tonclveng ss 4 
among the Ratees. 


and, 


daadinot rate. 
elise, 


one rater was absent. 
However, he was 
he was eouunted 


Because some ratees were marked "NO" by some raters, 


the total rating scores possible varied among ratees. 


epimameate this artifact, 


cio; 


average ratings were computed and 
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used in the later statistical analyses. This method for 
computing average ratings was also used with the ratings from 
the RS 1's. As there was only one RS 2, no computation of an 
average was necessary. 

The computations as illustrated in Figure II were also 
used to examine differences among raters. It was considered 
likely that some raters would be more "lenient" than others 
and would tend to give higher ratings. Figure II demonstrates 
this likelihood in the ranges of average ratings. Such rater 
differences could have had an unwanted influence on the data, 
because some raters had marked some ratees "NO." If, for 
instance, a ratee had been marked "NO" by all of the relatively 
lenient raters, he would have suffered a comparative dis- 
advantage when his ratings were compared with the ratings of 
other ratees. To overcome these difficulties, the ratings 
for each of the raters in the P and RS 1 sets were standardized 
to a mean of zero and a standard deviation of one for each 
scale and rater separately. Figure III shows how this 
Standardization was conducted. 

The intercorrelations among the ratings given by the peers 
were Computed using the raters' standardized ratings. Inter- 
WJreoelticnotsnanone P and RS 1 ratings of P's used the average 
Patines assigned to a P. Figure IV illustrates how these 
correlations were computed. The average P ratings were also 
used to compute the P and RS 2 correlations. Appendix B contains 
the average ratings used to compute all correlations in the 


meesent study. 





ES 


rreure Iii 


Example of Rating Standardization Procedure 


for the Peers 


(Data in Table 1 are standardized ratings) 


Scale 1 
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ineeene conduct of data collection, one rater was 


a0) 


Thus, he was counted 


For purposes of rating, he did not rate but 


absent. 


he was rated by all others. 


among the Ratees. 


‘not observed.' 


NO means 


be. 
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Figure IV 


Example of Intercorrelation Computation 
tom sPeers and Reporting Ssemior | 


(Data in Table are standardized ratings) 


Scale 1 
Average Rating Average Rating of 
Omer Dy Ommer P's P by RS 1's 
0.75 
2{ ooo 0.70 
3 0.13 WR 
! 0.55 
0.52 
Ratees 6 le hy 
0.70 
8 0.19 al 
9 -0.48 0.66 
10 0274 = LO 
1 2.02 
2 -0.22 -1.41 
Meese 020 


Analysis of the intercorrelations was made using the 
Wiheiewalt-miltirater technique (Campbell and Fiske, 1959; 
Lawler, 1967). The matrix of intercorrelations was analyzed 
in terms of convergent and discriminant validity, as these 
Validities furnish evidence of the construct validity of the 
Scales. Convergent validity reflects agreement among raters 
in evaluating dimensions of behavior. This validity is 
demonstrated by correlations between the same dimensions, as 


/maed by different raters, being significantly different 





Ibs 


from zero. Discriminant validity reflects the independence 
of the performance dimensions. Campbell and Fiske (1959) 
Suggest three criteria for demonstrating discriminant validity: 

1. Discriminant validity exists if the correlation be- 
tween raters for a dimension is higher than the correlation 
between that dimension and any other dimension which has 
neither dimension nor rater in common. 

2. Discriminant validity exists when a dimension cor- 
heomaces more highly with an independent effort to measure the 
Same dimension than with measures designed to assess different 
dimensions which employ the same rater. 

3. Discriminant validity exists when the same patterns 
of dimension intercorrelations exist for all common and 
different rater combinations. 

Thus, the study assessed the convergent validity among the 
raters (RS's and P's) when using the same scales and the dis- 
criminant validity of the three performance dimensions (Scale l, 
Semler. Scale’ 3) . 

fomdetermine the relaabaiaty of the raters’ ratings, an 
averaging procedure was used. Guilford states (1965) that 
Pitracornelations among waters are an indication” on tne re- 
miapility of ratings. lf ratings had@not been Missing in the 
pets of ratings from the P's and RS 11'S, intraclass correla- 
tions could have been easily computed and used as estimates of 
inter-rater agreement. Because of the missing observations, 
however, the actual correlations among all of the pairs of 
peer ratings were computed. The inter-rater correlations were 


piso computed for the pairs of RS l’s. 
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Momeonpite the average interpeer correlation, the inter- 
peer correlations were transformed using Fishers z's and 
pemomeca Dy their degrees of freedom (McNemar, 1962, p. 139 & 
£40). To estimate the reliability of the average ratings 
(averaged over raters) of the peers, the Spearman-Brown equa- 
tion (McNemar, 1962, p. 208) was used with the average 
met peexecOrrelation. This process was repeated for each 
Pomine three scales and for the P's and RS 1's. The estimates 
Peo erecitdapilities of the average ratings appear on the 
feta pility diagonal of the multitrait-multirater matrix. 

Results 

The multitrait-multirater matrix 1s shown in Table 1. 

The convergent validity diagonal (correlation coefficients 
Meparentheses) provided evidence that P's, RS 1's, and 
Soeeewere Mot in agreement in rating P's on the three scales. 
Agreement was substantial between P's and RS 1's on only one 
Beare, ocale Zz. 

With respect to discriminant validity, none of the 
criteria specified by Campbell and Fiske (1959) were met by 
Miemenesentumaata. With the exception of one case, all of 
the correlations in the monorater (solid line) triangles 
were high and significant. The exception was among the P's 
on S@ate 1 and 3. 

Since neither convergent nor discriminant validity were 
Sstablished in the present study, there is doubt as to the 
monstumet validity of the performance dimensions, and as to 


MiGet Pater aprecment. 





a 


TABLE A 


RELIABILITIES, INTERCORRELATIONS BETWEEN PEERS, REPORTING 
SENIORS FIRST LEVEL, AND REPORTING SENIOR SECOND LEVEL 
ON THREE SCALES @> 2, ¢ 





Raters RS 1 RS 2 





a. Convergent validity (monoscale-heterorater) coefficients 
age in parentheses, e.g., (-.20). 


b. Reliabilities for the averaged (over raters) ratings appear 
as unadorned correlation coefficients, e.g., .72 


G- sSoOlid-and dashed-line triangles contain the discriminant 
Moaladity (heteroscale-heterorater) coefficients. Solid- 
line triangles indicate monorater source, and dashed-line 
Eriameles indicate heterorater source. 





18 


The reliability diagonal (unadorned correlation coeffi- 
cients outside the monorater triangles) showed high 
reliabilities for RS 1's, reflecting at least their agreement 
with one another, and showed low reliabilities for P's, re- 
flecting disagreement among their ratings. 

The validity coefficients involving the P's and RS 1's 
Bepresent the correlations between the average P wratings 
peovided by those rater sets. 

Discussion 

imGenrrater reliabilities, shown along the reliability 
fagonal OLethe multitrait-multinater matrix, ranged from 
-10 to .94. Those for the RS 1 rater set were the highest. 
These reliabilities averaged around .41 for the P rater 
[ct wmpUiemuaimead tromes72 LorsSeale 1] to .10 for Scale 2. 

The RS 1 rater set had interrater agreements ranging from 
wooumtco .94. 

The convergent validities, shown along the validity 
diagonal, were usually negative, indicating not only no 
agmecmnemt, but also disagneememt among the different rating 
SOumecs. Ihe one exception to this fact was the .22 corre- 
lation between P and RS 1 on Scale 2. These validities 
Bamegederronec.47 to «22. 

Pew icerimanantevalidities in the mononater triangies 
(solid line) were positive. They ranged from .06 to 1.00. 
Those for the P rater set were the lowest, their average being 
areumas.44. and those for the RS 1 rater set were the highest, 
fmenmeatieaverage Of .86. The fact that these validities were 


meee ye Ro | Set and relatively high in the RS Z set 


_ — — 
— 
— 
_ 
— 
— 





1, 


probably is evidence of the halo effect in these two rating 
sources. 

The discriminant validities in the heterorater tri- 
angles (dashed-line) were usually negative. These validities 
Pompea trol 747 tO .c0 and averapeda around -.08. 

Uiewretereseale-neveroratemecorrerarions iImvolving the 
RS 2 and the P's were fairly large in magnitude and negative, 
Due NOt Statistically significamt, e.g., -.44, -.4is, and 
ae 

The major purpose of the present study was to examine 
peer evaluations in terms of convergent and discriminant 
woldity. “Ihe results ’revealledmanteres ting problems pemtratn - 
imgeweo Doth@the raters “and =the ratingescales: 

The multitrait-multirater analysis provided no evidence 
POm—comvereoMemvaliarey. With the exception of tie .cZ 
SOurelatronescocrricrent, ali convergent validity cockficrenus 
VomcmMlerativeuwratherutwan positives lhese results pointed 
out that there existed three widely different viewpoints 
among tme raters as to performamce of the P's. The results 
SCfethe present Study, tien, did not commcide with those of 
Hollander (1965), which presented evidence for validity and 
agreement with a .40 correlation between the initial peer 
ratings and the final performance criterion of ratings given 
the»peers by their superiors two years later. It should be 
noted that in this study all ratings were gathered at the 


same time in contrast to Hollander's study (1965). 
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With regard to the status of those during the evaluation 
relative to those being evaluated and the resulting different 
Viewpoints, two explanations are necessary. 

First, the two RS levels were not equivalent to each 
other in their opportunity to evaluate the P's. RS 1's had 
direct daily contact with at least three P's and supervised 
them closely. In some cases, a RS 1 may have had this close 
daily contact with the other nine P's. In the present study, 
then, each RS 1 was at least able to observe closely and 
evoaunete three P Ss plus some or all of the other maine P's. 

The RS 2, on the other hand, was farther fononed fOMeenie Ps. 
being involved in administrative functions and direct super- 
version Odmehe RS 1's -eeibusmthesRomzehadelessMopportunity. to 
observe closely and evaluate the P's. As a result of this 
difference in opportunity to evaluate, different perspectives 
between the RS 1's and the RS 2 may have caused them to expect, 
observe, or value different behaviors, and the lack of con- 
wengent validity would not, therefore, be surprising. 

Second, the P's were in a position to observe one another 
closely and so were relatively equivalent in their opportunity 
to evaluate one another. The P's had frequent and close 
COlmegict With Onewanotherameinclhenoutinessewere practically the 
same. Thus, as Leavitt and Williams pointed out (1947), the 
P's should be able to provide valid measures of one another's 
Demrormance. The reliabitiweyeatagonal of =the P*’s, however, 
did not support their ability to rate in agreement with one 


another. Apparently, different perspectives among themselves 
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and different perspectives from those of the RS's led the P's 
to expect or value different behaviors. If this interpretation 
1s true, the negative convergent validity of the P's with the 
RS's is not surprising. 

The one exception to the foregoing explanation occurred 
Memween P's and RS 1*s on Scale 2. Although the 522 cor- 
relation was positive and tended to show interrater agreement, 
He 1s not statistically sawenificamt (p > .05). 

The multitrait-multirater analysis also showed limited 
evidence for discriminant validity. This fact suggested 
that there was dependency between the scales. The results 
manifested this dependency in that almost all correlations 
in the monorater (solid line) triangles were high. These 
heotineowl Pohiohited= the présence OpeuEehin-pacer Seurcemmalc. 

The rating scales themselves previded the dimensions son 
which the raters were to evaluate the ratees. However, the 
Seales were likely perceived differently and the ratings in- 
fluenced by different rater values when used by the raters. 
he investagator had attempted to minimize the occurrence of 
this source of variance in the experimental design of the 
present study by meeting with all subjects to review the in- 
Structions contained on the evaluation form. 

Given, then, the lack ofsconvergent and the limited dis- 
Pinata validity, the results providedelittle»to no 
euia@ence@ef construct validity of thewrating seales. 

On a positive note, the results furnished evidence of 


two noteworthy points. First, the lack of convergent validity, 
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iieerar asethe P's were concerned, indicated that P's had 
Meimeseor View different from those of their RS's. This may 
support Williams and Leavitt's thesis (1947) which pointed 

out that peer evaluations are more valid than those given by 
ime peers  “SUpPeTIOT Ommeieets. = =15 ero Which ¢eroup. © Sor 
Paecreiad tne trues tacts ,"ynoOvONe wi livever =know put at 
Meactmeneer s viewpoints deservevconsideration.,  tJhere is a 
distracting fact here, however, as the interrater agreement 
among the P's was low. Second, the high agreement among RS l's, 
asmevadenced by their reliability Glaponel prevrded evidence 
that they may be in a better position to rate ratees than the 
RS 2. This interpretation supports Williams and Leavitt (1947), 
who pointed out that when there are two groups in a position 

to evaluate a third group, the evaluation group having the 
SuGceme association With=the third group would be expected to 
provide the more valid evaluations. 

Fimelly, two implications can be made from the results 
miamdicselssitom Neretotore presemued, First, with repard to 
Ene laek of convergent validity, Paseo bo Sean dmienic hose 
NoOlMmaniOt totally aeree with one another as to whom to promote 
and whom to select. In a task oriented organization like the 
NMGmiwemeomos, thismdlsagmecment Can potentially lead to in- 
decision, wavering, and improper personnel action, all of 
which would detract from effective job performance and effi- 
cient organizational operation. Second, in connection with 
the high agreement among RS 1's, even though there was dis- 


waueet@enmtsoctween them and thre P*s and the RS Z, RS 1*s appear 
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fowne ianeene better position to evaluate officially sub- 
Ordinates than does the RS 2. This implication does not fit 
in with the Marine Corps' evaluation system at the company 
level, where the RS 2 performs the official evaluation. The 
howcewdoes consider recommendations from RS 1's, but he has the 
ultimate decision on the evaluation as presently done. Under 
the existing performance evaluation system in the Marzrine 
Corps this imperfection can lead to a misrepresentation of 
facts and erroneous personnel appraisal both of which would 
detract from eftective personnel] adminastmatien and or- 
ganizational operations. 
Conclusions 

ErOMmCne Tesults Of the resent Seudy , tie investiodtor 
concludes that peer evaluations should not be used to 
SUupm@lement the Current Marine Gonps periormance evaluation 
system. The sample of Marines used furnished convincing 
evidence that peer evaluations were not reliable, the average 
DiPpeiiatenedohecenentescOctn CheMUmTeCDOTTeCG Canlien Delle 7.40 
Considering this fact, there is no sound basis then to use 
peer evaluations in an official sense. Unofficially, however, 
peer evaluations could serve as a management tool to gain 
insight into viewpoints of the varying levels with an organiza- 
ELOmweaoutme Study Showed different points of view between the 
Eesmand Ika S. 

Me ceond CONCIUSIOM Is senate, within the present organiza- 
Ppioinameset up for performance evaluation at the company level, 
Piemeupervisor closest to the individual being evaluated 


“iomlcmoce responsible for preparing the official evaluation 
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on the individual. The platoon commanders (RS 1's) then 
should write the fitness reports on squad leaders (P's) 
rather than the company commander (RS 2). In connection with 
this conclusion, considerable training will be necessary to 
acquaint potential platoon commanders with performance 
evaluation. 

A final conclusion is that the behaviorial dimensions 
(the scales) on the current fitness report should be closely 
examemed in terms of present job descriptions to ensure that 
they (the scales) do in fact capture behaviorial traits deemed 
essential to job performance. This need was demonstrated by 


the whole pattern of findings. 
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Appendix A 


Peer Evaluation 
Introduction 

You have been together with your fellow Squad Leaders for 
some time now. From this contact, you have formed certain 
impressions regarding their present and future successes as 
Marines. 

Considering these impressions, and carefully weighing 
the qualities of being a successful Marine, you are to 
evaluate your fellow Squad Leaders on the following dimensions: 

Regular Duties 

General Value to the Service 

Requirements in Time of War 
Forms have been provided for the evaluation. 

Pibaveoviois: OF YOURSEVALUAEION TARE TO BE USED POR RESEARCH 
PURPOSES ONLY, AND WILL NOT AFFECT YOUR OR YOUR FELLOW MARINES' 
CAREERS . 

Pee teks lial tom 
Part” Daowcci fic? ins tyueeions 

impeValuating your fellow Squad Leaders, you are to per- 
form the following steps: 

1. Consult the Squad Leader Roster which has been provided 
you, and draw a line through your name. 

Poaeevaluate each of the remagning mames on your list 


Moing One Peer Evaluation Form per name. In this evaluation, 
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a. enter Only the name of the individual you are 
evaluating in the space provided for on the Form, and 

b. evaluate this individual only on the following 
dimensions (a brief definition follows each dimension): 

Cl) Reetlar tere =e the jOpSewhich the 
individual must perform everyday in order to have his unit 
motivated, squared away, and ready to go, to include handling 
of troops in the field and in garrison carrying out with 
energy and resolution that which 1s believed to be reasonable, 
Meee. Orsduty, an@ caring for troops. 

(2) General Value to the Service - your estimate 
of how the Marine compares with all other Marines of the same 
grade, taking into account such factors as the capacity to 
handle jobs of increasing responsibility and preference for 
having the Marine in your unit. 

(3) Requirements in Time of War - if your unit 
were in a combat environment, to what degree would you want 
the Marine to be a member of your unit. 

Cen mankinegmeach Famine ions the abbreviations 
Shown in the marking boxes on the Form stand for the following: 

NO Not Observed (Insufficient opportunity to 
evaluate). 

UN Unsatisfactory (Unacceptable performance). 

BA Below Average (Below the generally accepted 
standard). 

AV Average (At the generally accepted standard). 

AA Above Average (Above the generally accepted 


Standard). 
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Peeexcecticont (lo the degree seldom achieved by 
others of the same grade). 
OS Outstanding (One of the clearly superior 
individuals of his grade). 
Peer Evaluation Form 
Part 1 


Name of the individual you are evaluating: 


Last Initials 


Dimensions: 
Regular Duties 


General Value to the Service 





Requirements in Time of War (Circle the best phrase that 


applies) 
Not Prerer ape be Particularly 
Observed Not Willing Glad Desire 
ie To 1X) Lo 


Have Have Have Have 
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Average Ratings Used to Compute Correlations 
Table 1 


ivenracorrelationmetor Peers on Scales 1 and 2 
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lnmerecorrelation for Peers on Scales 1 and 3 


Average Rating o Memace “Kang oO 

P by Otmer Ps on P Dystener Ps son 
ocale | Scare 3 
54 
16 
“a8 
4 = gilt 
“96 
Ratees 6 == aul =, INS 
“2 
2 
8 ee =a 
10 74 mois 
Ja. = Woe Sot 
12 = eee = Se 





Table 3 


Intracorrelation for Peers on Scales 2 and 3 
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Intercorrelation for Peers and Reporting Senior l 
on Scale l 
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Table 5 


Intercorrelation for Peers on Scale 1 and 
Reporting Senmaor 1 on Scale 2 
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Imeercorrelation for’Peéers on Scale 1 and 
Reporting Senior 1 on Scale 3 
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intercomrelation foumsPecrs onvScale 2 <and 
Reporting senior Ijomescale: 1 
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Intercorrelation for Peers on Scale 2 and 
Reporting Senior 1 on Scale 2 
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Table 9 


Intercorrelation for Peers on Scale 2 and 
Reporting Senior 1 on Scale 3 
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Intercorrelation for Peers on Scale 3 and 
Reposeing Senior 1 on Scale 1 
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Inter@errelation for Peers on Scale 3 and 
Reportine occnior 1 on Seale Z 
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Table 12 


Intercorrelation for Peers on Scale 3 and 
Reporting Senior 1 on Scale 3 
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Intracorrelation for Reporting Senior 1's 
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Intracorrelation for Reporting Senior 1's 
Oonwocales leg 3 
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Intercorrelation for Peers on Scale 1 
ana Reporting Sen#erm 2 on Seale 1 
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Intercorrelation for Peers on Scale 1 
and Reporting Senior 2 on Scale 3 
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Intercorrelation for Peers on Scale 2 
and Reporting Senior 2 on Scale 1 
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Intemeorrelation for Peers on Scale 2 
and Reporting Senior 2 on Scale 2 
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Intercorrelation for Peers on Scale Z 
and Reporting Senior 2 on Scale 3 
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Intercorrelation for Peers on Scale 3 
and Reporting Senior 2 on Scale 1 
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iMtucwcorrelation for Peers on Scale 3 
and Reporting Senior Z on Scale 2 
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Intercorrelation for Peers on Scale 3 
and Reporting Senior 2 on Scale 3 
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Intercorrelation for Reporting Senior 1's on Scale l 
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Intercorrelation for Reporting Senior 1's on Scale 1 
and Reporting Senior 2 on Scale 2 
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ieeercorrelation for Reporting Senior 1's on Scale 2 
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Dviewage .Kating Or P Rating of P by 

by RS 1*s on Scale 3 Romeo socale | 
it Ls All 
2 “1.04 
“1.04 
“1.04 
1s 
patee 6 | = 90 “1,04 
1.45 
8 = es -1.04 
: 20 
1 20 
 - e 20 
to 1.45 





Table 32 


Intercorrelation for Reporting Senior 1's on Scale 3 
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